Skip to content

[Feature] Tool services #3220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Nov 6, 2025
Merged

[Feature] Tool services #3220

merged 8 commits into from
Nov 6, 2025
+4,529 −261

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 23, 2025 •
edited
Loading

@vmoens
Update
7f168a7 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 23, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3220

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 21 Pending, 9 Unrelated Failures

As of commit ef988e1 with merge base 13434eb (image):

NEW FAILURE - The following job has failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 23, 2025
vmoens added a commit that referenced this pull request Oct 23, 2025
@vmoens
[Feature] Tool services
33a1141 
ghstack-source-id: acf8de0
Pull-Request: #3220
This was referenced Oct 23, 2025
Merged
Merged
Merged
Merged
@github-actions
Copy link

github-actions bot commented Oct 23, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}19$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 84.0501μs 82.7178μs 12.0893 KOps/s 12.2033 KOps/s $\color{#d91a1a}-0.93\%$
test_tensor_to_bytestream_speed[torch.save] 0.1416ms 0.1413ms 7.0751 KOps/s 6.8596 KOps/s $\color{#35bf28}+3.14\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1266s 0.1261s 7.9284 Ops/s 8.1037 Ops/s $\color{#d91a1a}-2.16\%$
test_tensor_to_bytestream_speed[numpy] 2.9205μs 2.9194μs 342.5354 KOps/s 339.5324 KOps/s $\color{#35bf28}+0.88\%$
test_tensor_to_bytestream_speed[safetensors] 43.6010μs 43.4708μs 23.0039 KOps/s 23.1189 KOps/s $\color{#d91a1a}-0.50\%$
test_simple 0.6653s 0.5766s 1.7344 Ops/s 1.7367 Ops/s $\color{#d91a1a}-0.13\%$
test_transformed 1.2326s 1.1459s 0.8727 Ops/s 0.8756 Ops/s $\color{#d91a1a}-0.34\%$
test_serial 1.6753s 1.6728s 0.5978 Ops/s 0.5904 Ops/s $\color{#35bf28}+1.25\%$
test_parallel 1.0988s 1.0767s 0.9288 Ops/s 0.9052 Ops/s $\color{#35bf28}+2.61\%$
test_step_mdp_speed[True-True-True-True-True] 0.2249ms 45.0192μs 22.2128 KOps/s 22.5941 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[True-True-True-True-False] 52.4310μs 25.2914μs 39.5391 KOps/s 38.5284 KOps/s $\color{#35bf28}+2.62\%$
test_step_mdp_speed[True-True-True-False-True] 58.3110μs 24.8738μs 40.2029 KOps/s 38.7657 KOps/s $\color{#35bf28}+3.71\%$
test_step_mdp_speed[True-True-True-False-False] 41.6410μs 13.8461μs 72.2227 KOps/s 69.0117 KOps/s $\color{#35bf28}+4.65\%$
test_step_mdp_speed[True-True-False-True-True] 76.7410μs 47.8419μs 20.9022 KOps/s 21.0602 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-False-True-False] 55.0210μs 28.0904μs 35.5993 KOps/s 35.7852 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-True-False-False-True] 65.4510μs 28.0979μs 35.5899 KOps/s 35.1003 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-True-False-False-False] 46.1210μs 16.7731μs 59.6192 KOps/s 58.8513 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-False-True-True-True] 82.1110μs 51.2029μs 19.5302 KOps/s 19.4674 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-False-True-True-False] 69.6310μs 30.4722μs 32.8168 KOps/s 32.5555 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-False-True-False-True] 58.5710μs 27.7072μs 36.0917 KOps/s 34.1641 KOps/s $\textbf{\color{#35bf28}+5.64\%}$
test_step_mdp_speed[True-False-True-False-False] 45.9810μs 16.5317μs 60.4899 KOps/s 58.8311 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[True-False-False-True-True] 81.5810μs 53.0981μs 18.8331 KOps/s 18.9927 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-False-False-True-False] 56.6010μs 33.0775μs 30.2321 KOps/s 30.0240 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-False-False-True] 60.3410μs 30.3137μs 32.9883 KOps/s 33.0415 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-False-False-False-False] 42.9000μs 19.2624μs 51.9145 KOps/s 51.3905 KOps/s $\color{#35bf28}+1.02\%$
test_step_mdp_speed[False-True-True-True-True] 0.1063ms 49.1606μs 20.3415 KOps/s 20.1787 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-True-True-True-False] 53.8600μs 30.7302μs 32.5413 KOps/s 32.8388 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[False-True-True-False-True] 2.3994ms 31.8342μs 31.4127 KOps/s 30.4318 KOps/s $\color{#35bf28}+3.22\%$
test_step_mdp_speed[False-True-True-False-False] 45.4110μs 18.5025μs 54.0469 KOps/s 54.2133 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[False-True-False-True-True] 86.1810μs 53.2150μs 18.7917 KOps/s 18.6994 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-True-False-True-False] 63.2310μs 33.2655μs 30.0612 KOps/s 29.8997 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-True-False-False-True] 67.7720μs 34.2586μs 29.1897 KOps/s 28.5715 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[False-True-False-False-False] 51.1700μs 21.1075μs 47.3766 KOps/s 47.4782 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-False-True-True-True] 0.1044ms 56.6713μs 17.6456 KOps/s 18.0789 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[False-False-True-True-False] 68.0810μs 36.5042μs 27.3941 KOps/s 27.6154 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-False-True-False-True] 66.1210μs 34.1733μs 29.2626 KOps/s 29.0800 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-False-True-False-False] 47.3810μs 20.8633μs 47.9311 KOps/s 47.2195 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[False-False-False-True-True] 99.1420μs 58.0209μs 17.2352 KOps/s 17.1813 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-False-True-False] 70.8620μs 38.3407μs 26.0819 KOps/s 25.6272 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[False-False-False-False-True] 74.4720μs 36.2153μs 27.6127 KOps/s 26.7231 KOps/s $\color{#35bf28}+3.33\%$
test_step_mdp_speed[False-False-False-False-False] 57.3710μs 23.7792μs 42.0535 KOps/s 42.3046 KOps/s $\color{#d91a1a}-0.59\%$
test_values[generalized_advantage_estimate-True-True] 10.3292ms 10.0353ms 99.6479 Ops/s 100.8923 Ops/s $\color{#d91a1a}-1.23\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.5852ms 17.5887ms 56.8546 Ops/s 57.4240 Ops/s $\color{#d91a1a}-0.99\%$
test_values[td0_return_estimate-False-False] 0.2077ms 0.1311ms 7.6262 KOps/s 7.6984 KOps/s $\color{#d91a1a}-0.94\%$
test_values[td1_return_estimate-False-False] 28.8239ms 27.9507ms 35.7772 Ops/s 35.7887 Ops/s $\color{#d91a1a}-0.03\%$
test_values[vec_td1_return_estimate-False-False] 18.8048ms 17.7071ms 56.4746 Ops/s 56.9605 Ops/s $\color{#d91a1a}-0.85\%$
test_values[td_lambda_return_estimate-True-False] 42.1256ms 41.6036ms 24.0364 Ops/s 24.0688 Ops/s $\color{#d91a1a}-0.13\%$
test_values[vec_td_lambda_return_estimate-True-False] 18.2382ms 17.6775ms 56.5690 Ops/s 56.9296 Ops/s $\color{#d91a1a}-0.63\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.8576ms 8.6620ms 115.4462 Ops/s 115.7767 Ops/s $\color{#d91a1a}-0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.6951ms 1.4692ms 680.6309 Ops/s 672.2987 Ops/s $\color{#35bf28}+1.24\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5353ms 0.4250ms 2.3532 KOps/s 2.3586 KOps/s $\color{#d91a1a}-0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.7715ms 34.2448ms 29.2015 Ops/s 28.9766 Ops/s $\color{#35bf28}+0.78\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.2380ms 1.7047ms 586.6169 Ops/s 575.9217 Ops/s $\color{#35bf28}+1.86\%$
test_dqn_speed[False-None] 6.4578ms 1.4396ms 694.6184 Ops/s 695.1515 Ops/s $\color{#d91a1a}-0.08\%$
test_dqn_speed[False-backward] 2.0116ms 1.9585ms 510.6014 Ops/s 514.2291 Ops/s $\color{#d91a1a}-0.71\%$
test_dqn_speed[True-None] 0.9134ms 0.5200ms 1.9232 KOps/s 1.9052 KOps/s $\color{#35bf28}+0.95\%$
test_dqn_speed[True-backward] 1.0295ms 0.9710ms 1.0299 KOps/s 1.0029 KOps/s $\color{#35bf28}+2.69\%$
test_dqn_speed[reduce-overhead-None] 0.9093ms 0.5111ms 1.9565 KOps/s 1.9378 KOps/s $\color{#35bf28}+0.96\%$
test_dqn_speed[reduce-overhead-backward] 1.3431ms 0.9632ms 1.0382 KOps/s 1.0321 KOps/s $\color{#35bf28}+0.60\%$
test_ddpg_speed[False-None] 3.1676ms 2.8807ms 347.1411 Ops/s 342.5123 Ops/s $\color{#35bf28}+1.35\%$
test_ddpg_speed[False-backward] 4.2715ms 4.1368ms 241.7302 Ops/s 241.4536 Ops/s $\color{#35bf28}+0.11\%$
test_ddpg_speed[True-None] 1.7625ms 1.3904ms 719.1934 Ops/s 717.0704 Ops/s $\color{#35bf28}+0.30\%$
test_ddpg_speed[True-backward] 2.4580ms 2.3868ms 418.9718 Ops/s 379.0059 Ops/s $\textbf{\color{#35bf28}+10.54\%}$
test_ddpg_speed[reduce-overhead-None] 2.0577ms 1.3866ms 721.2000 Ops/s 707.0365 Ops/s $\color{#35bf28}+2.00\%$
test_ddpg_speed[reduce-overhead-backward] 2.4197ms 2.3662ms 422.6134 Ops/s 406.5718 Ops/s $\color{#35bf28}+3.95\%$
test_sac_speed[False-None] 8.4679ms 7.9647ms 125.5534 Ops/s 126.9195 Ops/s $\color{#d91a1a}-1.08\%$
test_sac_speed[False-backward] 11.6731ms 11.3058ms 88.4503 Ops/s 88.5838 Ops/s $\color{#d91a1a}-0.15\%$
test_sac_speed[True-None] 2.3097ms 2.1211ms 471.4436 Ops/s 466.9567 Ops/s $\color{#35bf28}+0.96\%$
test_sac_speed[True-backward] 4.2798ms 4.0784ms 245.1921 Ops/s 231.0339 Ops/s $\textbf{\color{#35bf28}+6.13\%}$
test_sac_speed[reduce-overhead-None] 2.2948ms 2.1154ms 472.7142 Ops/s 454.2889 Ops/s $\color{#35bf28}+4.06\%$
test_sac_speed[reduce-overhead-backward] 4.1653ms 4.0738ms 245.4740 Ops/s 225.8906 Ops/s $\textbf{\color{#35bf28}+8.67\%}$
test_redq_speed[False-None] 10.7335ms 10.2918ms 97.1651 Ops/s 82.3857 Ops/s $\textbf{\color{#35bf28}+17.94\%}$
test_redq_speed[False-backward] 23.8914ms 18.7300ms 53.3902 Ops/s 54.9425 Ops/s $\color{#d91a1a}-2.83\%$
test_redq_speed[True-None] 4.8931ms 4.4234ms 226.0679 Ops/s 227.4967 Ops/s $\color{#d91a1a}-0.63\%$
test_redq_speed[True-backward] 10.1764ms 9.9004ms 101.0063 Ops/s 101.0556 Ops/s $\color{#d91a1a}-0.05\%$
test_redq_speed[reduce-overhead-None] 4.5206ms 4.3735ms 228.6486 Ops/s 227.5160 Ops/s $\color{#35bf28}+0.50\%$
test_redq_speed[reduce-overhead-backward] 10.5242ms 10.1652ms 98.3750 Ops/s 99.4337 Ops/s $\color{#d91a1a}-1.06\%$
test_redq_deprec_speed[False-None] 11.3565ms 11.0361ms 90.6119 Ops/s 88.7172 Ops/s $\color{#35bf28}+2.14\%$
test_redq_deprec_speed[False-backward] 16.5614ms 16.0411ms 62.3397 Ops/s 61.8913 Ops/s $\color{#35bf28}+0.72\%$
test_redq_deprec_speed[True-None] 4.0396ms 3.6302ms 275.4685 Ops/s 272.5960 Ops/s $\color{#35bf28}+1.05\%$
test_redq_deprec_speed[True-backward] 7.9459ms 7.6966ms 129.9279 Ops/s 135.3957 Ops/s $\color{#d91a1a}-4.04\%$
test_redq_deprec_speed[reduce-overhead-None] 3.9753ms 3.5881ms 278.7007 Ops/s 257.6465 Ops/s $\textbf{\color{#35bf28}+8.17\%}$
test_redq_deprec_speed[reduce-overhead-backward] 8.2086ms 7.7039ms 129.8045 Ops/s 118.5653 Ops/s $\textbf{\color{#35bf28}+9.48\%}$
test_td3_speed[False-None] 8.0930ms 7.9460ms 125.8493 Ops/s 125.9215 Ops/s $\color{#d91a1a}-0.06\%$
test_td3_speed[False-backward] 12.0457ms 10.9361ms 91.4405 Ops/s 92.1627 Ops/s $\color{#d91a1a}-0.78\%$
test_td3_speed[True-None] 1.8731ms 1.8048ms 554.0642 Ops/s 540.8913 Ops/s $\color{#35bf28}+2.44\%$
test_td3_speed[True-backward] 4.6864ms 3.6660ms 272.7773 Ops/s 269.4942 Ops/s $\color{#35bf28}+1.22\%$
test_td3_speed[reduce-overhead-None] 1.8171ms 1.7780ms 562.4414 Ops/s 559.9228 Ops/s $\color{#35bf28}+0.45\%$
test_td3_speed[reduce-overhead-backward] 4.0676ms 3.7007ms 270.2192 Ops/s 277.1972 Ops/s $\color{#d91a1a}-2.52\%$
test_cql_speed[False-None] 28.8866ms 26.3446ms 37.9584 Ops/s 38.3672 Ops/s $\color{#d91a1a}-1.07\%$
test_cql_speed[False-backward] 40.8108ms 36.2144ms 27.6134 Ops/s 27.9020 Ops/s $\color{#d91a1a}-1.03\%$
test_cql_speed[True-None] 12.5148ms 12.2814ms 81.4237 Ops/s 81.1801 Ops/s $\color{#35bf28}+0.30\%$
test_cql_speed[True-backward] 18.7503ms 18.3754ms 54.4205 Ops/s 56.3047 Ops/s $\color{#d91a1a}-3.35\%$
test_cql_speed[reduce-overhead-None] 12.8470ms 12.4654ms 80.2218 Ops/s 78.8307 Ops/s $\color{#35bf28}+1.76\%$
test_cql_speed[reduce-overhead-backward] 18.9908ms 18.4818ms 54.1074 Ops/s 56.5371 Ops/s $\color{#d91a1a}-4.30\%$
test_a2c_speed[False-None] 6.2198ms 5.3386ms 187.3159 Ops/s 178.8293 Ops/s $\color{#35bf28}+4.75\%$
test_a2c_speed[False-backward] 12.2800ms 11.8983ms 84.0455 Ops/s 82.8959 Ops/s $\color{#35bf28}+1.39\%$
test_a2c_speed[True-None] 3.8824ms 3.7432ms 267.1535 Ops/s 263.0620 Ops/s $\color{#35bf28}+1.56\%$
test_a2c_speed[True-backward] 8.9057ms 8.7332ms 114.5060 Ops/s 111.3303 Ops/s $\color{#35bf28}+2.85\%$
test_a2c_speed[reduce-overhead-None] 4.0230ms 3.7165ms 269.0691 Ops/s 264.6217 Ops/s $\color{#35bf28}+1.68\%$
test_a2c_speed[reduce-overhead-backward] 9.0537ms 8.8982ms 112.3825 Ops/s 109.9094 Ops/s $\color{#35bf28}+2.25\%$
test_ppo_speed[False-None] 6.1295ms 5.8391ms 171.2606 Ops/s 166.3727 Ops/s $\color{#35bf28}+2.94\%$
test_ppo_speed[False-backward] 12.7982ms 12.4411ms 80.3789 Ops/s 78.3567 Ops/s $\color{#35bf28}+2.58\%$
test_ppo_speed[True-None] 3.9917ms 3.6424ms 274.5420 Ops/s 262.3123 Ops/s $\color{#35bf28}+4.66\%$
test_ppo_speed[True-backward] 8.7622ms 8.5926ms 116.3790 Ops/s 112.4965 Ops/s $\color{#35bf28}+3.45\%$
test_ppo_speed[reduce-overhead-None] 3.8611ms 3.6392ms 274.7836 Ops/s 264.5423 Ops/s $\color{#35bf28}+3.87\%$
test_ppo_speed[reduce-overhead-backward] 8.8743ms 8.7681ms 114.0498 Ops/s 107.2599 Ops/s $\textbf{\color{#35bf28}+6.33\%}$
test_reinforce_speed[False-None] 4.8691ms 4.6315ms 215.9129 Ops/s 213.4781 Ops/s $\color{#35bf28}+1.14\%$
test_reinforce_speed[False-backward] 7.6879ms 7.4862ms 133.5797 Ops/s 131.9297 Ops/s $\color{#35bf28}+1.25\%$
test_reinforce_speed[True-None] 3.0584ms 2.8646ms 349.0887 Ops/s 340.7354 Ops/s $\color{#35bf28}+2.45\%$
test_reinforce_speed[True-backward] 8.2325ms 7.7377ms 129.2371 Ops/s 119.6297 Ops/s $\textbf{\color{#35bf28}+8.03\%}$
test_reinforce_speed[reduce-overhead-None] 2.9934ms 2.8618ms 349.4249 Ops/s 333.7655 Ops/s $\color{#35bf28}+4.69\%$
test_reinforce_speed[reduce-overhead-backward] 8.0714ms 7.8978ms 126.6168 Ops/s 119.4373 Ops/s $\textbf{\color{#35bf28}+6.01\%}$
test_iql_speed[False-None] 20.7059ms 20.0584ms 49.8544 Ops/s 50.9000 Ops/s $\color{#d91a1a}-2.05\%$
test_iql_speed[False-backward] 36.6969ms 31.2385ms 32.0118 Ops/s 32.3719 Ops/s $\color{#d91a1a}-1.11\%$
test_iql_speed[True-None] 9.2155ms 8.5166ms 117.4180 Ops/s 115.6283 Ops/s $\color{#35bf28}+1.55\%$
test_iql_speed[True-backward] 17.4089ms 16.8357ms 59.3975 Ops/s 59.2216 Ops/s $\color{#35bf28}+0.30\%$
test_iql_speed[reduce-overhead-None] 9.1581ms 8.5834ms 116.5042 Ops/s 112.7984 Ops/s $\color{#35bf28}+3.29\%$
test_iql_speed[reduce-overhead-backward] 17.9307ms 17.3316ms 57.6981 Ops/s 57.1361 Ops/s $\color{#35bf28}+0.98\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4798ms 6.0599ms 165.0203 Ops/s 102.5259 Ops/s $\textbf{\color{#35bf28}+60.95\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6576ms 0.3087ms 3.2392 KOps/s 2.8590 KOps/s $\textbf{\color{#35bf28}+13.30\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5367ms 0.2649ms 3.7745 KOps/s 3.0358 KOps/s $\textbf{\color{#35bf28}+24.33\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0011ms 5.7762ms 173.1243 Ops/s 173.8161 Ops/s $\color{#d91a1a}-0.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8853ms 0.3168ms 3.1568 KOps/s 3.1085 KOps/s $\color{#35bf28}+1.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5030ms 0.3071ms 3.2561 KOps/s 3.0047 KOps/s $\textbf{\color{#35bf28}+8.37\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5017ms 1.3139ms 761.0859 Ops/s 786.7754 Ops/s $\color{#d91a1a}-3.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4293ms 1.2392ms 806.9548 Ops/s 858.7651 Ops/s $\textbf{\color{#d91a1a}-6.03\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0593ms 5.8998ms 169.4971 Ops/s 167.3572 Ops/s $\color{#35bf28}+1.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.5070ms 0.4586ms 2.1807 KOps/s 2.2897 KOps/s $\color{#d91a1a}-4.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8743ms 0.4346ms 2.3012 KOps/s 2.3343 KOps/s $\color{#d91a1a}-1.42\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9131ms 5.7762ms 173.1240 Ops/s 172.4950 Ops/s $\color{#35bf28}+0.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.2295ms 0.2988ms 3.3470 KOps/s 2.9562 KOps/s $\textbf{\color{#35bf28}+13.22\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5586ms 0.3551ms 2.8161 KOps/s 3.8514 KOps/s $\textbf{\color{#d91a1a}-26.88\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9931ms 5.7391ms 174.2420 Ops/s 173.5616 Ops/s $\color{#35bf28}+0.39\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8719ms 0.3671ms 2.7244 KOps/s 3.4937 KOps/s $\textbf{\color{#d91a1a}-22.02\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7263ms 0.3471ms 2.8813 KOps/s 3.0514 KOps/s $\textbf{\color{#d91a1a}-5.57\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0072ms 5.9221ms 168.8588 Ops/s 167.6750 Ops/s $\color{#35bf28}+0.71\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0743ms 0.4968ms 2.0130 KOps/s 2.0157 KOps/s $\color{#d91a1a}-0.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6226ms 0.4127ms 2.4230 KOps/s 2.1825 KOps/s $\textbf{\color{#35bf28}+11.02\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4957s 14.8109ms 67.5178 Ops/s 56.5013 Ops/s $\textbf{\color{#35bf28}+19.50\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.1737ms 1.9175ms 521.5164 Ops/s 712.7791 Ops/s $\textbf{\color{#d91a1a}-26.83\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.9306ms 1.1563ms 864.7933 Ops/s 967.2330 Ops/s $\textbf{\color{#d91a1a}-10.59\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.8499ms 5.0603ms 197.6157 Ops/s 194.6981 Ops/s $\color{#35bf28}+1.50\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.5734ms 2.0240ms 494.0688 Ops/s 491.8080 Ops/s $\color{#35bf28}+0.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.0960ms 1.1914ms 839.3195 Ops/s 827.0340 Ops/s $\color{#35bf28}+1.49\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4145s 13.4274ms 74.4746 Ops/s 60.2941 Ops/s $\textbf{\color{#35bf28}+23.52\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.5165ms 2.0922ms 477.9663 Ops/s 563.6465 Ops/s $\textbf{\color{#d91a1a}-15.20\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.1933ms 0.9961ms 1.0039 KOps/s 736.7306 Ops/s $\textbf{\color{#35bf28}+36.27\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.4474ms 32.6913ms 30.5892 Ops/s 30.3324 Ops/s $\color{#35bf28}+0.85\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.1130ms 17.5160ms 57.0905 Ops/s 57.8288 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 35.6446ms 33.7191ms 29.6568 Ops/s 28.9403 Ops/s $\color{#35bf28}+2.48\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.3176ms 17.7300ms 56.4016 Ops/s 56.9217 Ops/s $\color{#d91a1a}-0.91\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 40.3077ms 35.4671ms 28.1952 Ops/s 28.2356 Ops/s $\color{#d91a1a}-0.14\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.4439ms 19.0217ms 52.5715 Ops/s 53.1484 Ops/s $\color{#d91a1a}-1.09\%$

@vmoens
Update
8722fe9 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Oct 23, 2025
@vmoens
[Feature] Tool services
f2164e9 
ghstack-source-id: 6863f90
Pull-Request: #3220
vmoens added a commit that referenced this pull request Oct 23, 2025
@vmoens
[Feature] Tool services
cc31c1e 
ghstack-source-id: 6863f90
Pull-Request: #3220
@vmoens
Update
da82d11 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Oct 23, 2025
@vmoens
[Feature] Tool services
1ee24cf 
ghstack-source-id: f444d4d
Pull-Request: #3220
@vmoens
Update
8022360 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Oct 25, 2025
@vmoens
[Feature] Tool services
f11c956 
ghstack-source-id: a6cdf57
Pull-Request: #3220
@vmoens
Update
fd322d9 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Oct 25, 2025
@vmoens
[Feature] Tool services
a3274be 
ghstack-source-id: bbad726
Pull-Request: #3220
@vmoens vmoens mentioned this pull request Oct 26, 2025
Closed
@vmoens
Update
d747f8e 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Oct 27, 2025
@vmoens
[Feature] Tool services
1610fe0 
ghstack-source-id: a95a346
Pull-Request: #3220
@vmoens
Update
6181945 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Oct 28, 2025
@vmoens
[Feature] Tool services
7f2bf09 
ghstack-source-id: 8e958b8
Pull-Request: #3220
@vmoens
Update
ef988e1 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Nov 6, 2025
@vmoens
[Feature] Tool services
24bdef9 
ghstack-source-id: 741581c
Pull-Request: #3220
@vmoens vmoens added enhancement New feature or request llm/feature labels Nov 6, 2025
vmoens added a commit that referenced this pull request Nov 6, 2025
@vmoens
[Feature] Tool services
9ca0e40 
ghstack-source-id: 741581c
Pull-Request: #3220
@vmoens vmoens merged commit ef988e1 into gh/vmoens/169/base Nov 6, 2025
59 of 79 checks passed
@vmoens vmoens deleted the gh/vmoens/169/head branch November 6, 2025 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request llm/feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens